The invention relates to a method and a device for three-dimensional reconstruction of a scene, and more particularly to a method and a device for determining spatial correspondences between image areas in a number of images forming at least two image sequences of a scene that are recorded from different observation perspectives.
Passive methods for three-dimensional scene reconstruction by means of image data are generally based on the determination of spatial correspondences between a number of images of the scene recorded from various directions and distances. This determination of correspondences corresponds to an identification of pixel positions or pixel areas in the images with points or objects or object sections in the scene to be reconstructed. After the correspondences are determined, the three-dimensional structure of the scene is usually determined in the form of a point cloud by means of known methods, with each point or each point concentration of the point cloud corresponding to the mutually assigned and corresponding pixel positions or pixel areas, respectively.
The procedure for determining these spatial correspondences is usually termed as the correspondence problem in the literature, and various approaches for solving this problem have been proposed:
A customary method for determining the spatial correspondences is the correlation of the image contents on a local plane, wherein contents of spatial windows are mutually compared by applying suitable error measures as similarity measure such as, for example, cross correlation coefficients, sum of the squared differences, or sum of the absolute differences. This method is temporally effective, in particular for calibrated stereo image pairs, i.e. image pairs for which only pixels lying on common epipolar lines can be in correspondence.
In a publication of J. Davis, D. Nehab, R. Ramamoorthi, S. Rusinkiewicz “Spacetime Stereo: A unifying framework for Depth from Triangulation”, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 27, No. 2, 2005, it is proposed to use not only the contents of spatial windows for correlation analysis, but to extend the correlation window into the time dimension. The sum of the squared differences of the spatial temporal correlation windows is used as similarity measure for forming correspondence.
Another approach for solving the correspondence problem is described in a publication of C. Wöhler, L. Krüger “A Contour based Stereo Vision Algorithm for Video Surveillance Applications”, SPIE Visual Communication and Image Processing, Lugano, 2003. In this approach, relevant image regions are determined in the images in a first step. Two-dimensional objects in the images are derived from the relevant image regions, and associated object contour lines are drawn. The correspondences are then determined by comparing the contour properties of the object contour lines at the intersection points with the epipolar lines.